Systems and methods of directing a lighting fixture in a venue

ABSTRACT

Methods and systems are provided for directing a lighting fixture in a venue. A user device is paired with a three-dimensional model space that represents a physical space of the venue. A lighting beam destination is indicated with the user device using a position and an orientation of the user device. The indicated lighting beam destination is converted into coordinates in the three-dimensional model space. A lighting fixture orientation value to direct a lighting beam of the lighting fixture to the lighting beam destination in the physical space of the venue is determined based on the coordinates of the lighting beam destination in the three-dimensional model space of the venue. A motor associated with the lighting fixture is actuated to move the lighting fixture to a lighting fixture orientation in the physical space of the venue based on the lighting fixture orientation values.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional PatentApplication No. 62/777,466, filed on Dec. 10, 2018, which is herebyincorporated by reference in its entirety.

FIELD

Embodiments described herein relate to directing or controlling alighting fixture in a venue.

SUMMARY

Systems and methods described herein relate to rendering lightingvisuals in a virtual reality or augmented reality interactiveenvironment as a way to allow a user to experiment with and discover thelighting visuals (e.g., a light beam, a lighting transition, a followspot, etc.) available for a given venue and/or lighting fixturearrangement. Three-dimensional models of the potential locations oflighting beams for given lighting fixtures are created and madeavailable in the interactive environment. These models includethree-dimensional representations of lighting beams and other lightingvisuals, or the models can be used as a three-dimensional model spacebounding the possible lighting beam destinations for a given lightingfixture in the real world. In some embodiments, the user directs one ormore lighting fixtures with the aid of a user device by indicating adesired lighting beam destination.

One component of accurately rendering and controlling lighting visualsin an interactive environment is having accurate information regardingthe lighting fixtures. Particularly, the arrangement information (e.g.,the position and orientation) of each lighting fixture should be knownwith accuracy. For example, one inch of discrepancy between the positiondata and the real-world position of the lighting fixture can cause theprojected lighting beam to be multiple feet away from the intendedlighting beam destination. Similarly, a discrepancy of a few degreesbetween the orientation data and the real-world orientation of thelighting fixture can cause a similar inconsistency.

Currently, a user must make precise measurements to accurately determinethe position of a lighting fixture. To determine orientation, the usermust also have specific knowledge of the lighting fixture and itscurrent hanging position. Because lighting fixtures can be mounted torafters, scaffolding, pipes, or any other structure in any number ofways, this technique requires significant time and effort to complete.Additionally, the user must be experienced and/or technically trained inorder to properly use the measurement tools and techniques. Aspecialized technician is often required. Also, these techniques areprone to human error, which can only be detected when a user attempts tocontrol the one or more lighting fixtures according to a desiredlighting visual. Once the discrepancy is discovered, which can be at alate hour in the setup process at a venue, the user must perform theentire measurement process again for the incorrectly calibrated light.If a specialized technician was contracted, the same or anotherspecialized technician may need to perform the process on short notice.Because the time of the event at the venue often has a firm deadline,this process can be stressful, inconvenient, and expensive.

Also currently, directing and focusing a light on a specific location ina venue requires a lighting programmer to adjust the lighting fixture'span and tilt until the correct location of the lighting beam isachieved. This process requires a skilled user and can be time consumingdue to the large number of lights at many venues. An example of the timeand effort involved includes setting up one or more focus palettes.Focus palettes are used in lighting design to define a specific set ofpositions upon which multiple moving lighting fixtures focus. Creatingand updating focus palettes can be very monotonous and can take aconsiderable amount of time. For example, a touring band will tour withhundreds of moving lighting fixtures and the show will require focuspalettes for each band member's multiple positions on stage throughoutthe show. At each new venue of the tour, every moving lighting fixturemust be moved and focused for at least one of the focus palettes. Thissetup process takes significant time during setup, which can bestressful for the user, expensive, and prone to human errors.

To address the above concerns, embodiments described herein providesystems and methods for determining the arrangement information of alighting fixture. Embodiments described herein determine the pertinentdetails about a lighting fixture's arrangement information (e.g.,location, mounting style, etc.) without requiring expensive measuringtools, expert knowledge, or a significant amount of time.

Systems and methods are described herein for directing a lightingfixture in a venue. Embodiments described herein greatly reduce theamount of time to set up and adjust lighting fixtures for tasks, such ascreating focus palettes and following a mark on a stage, withoutrequiring expert knowledge.

Methods are described herein for directing a lighting fixture in avenue. The method includes pairing a user device with athree-dimensional model space that represents a physical space of thevenue. A lighting beam destination in the physical space of the venue isindicated with the user device using a position and orientation of theuser device. An electronic processor converts the indicated lightingbeam destination into coordinates in the three-dimensional model space.The electronic processor determines a lighting fixture orientation valueto direct a lighting beam of the lighting fixture to the lighting beamdestination in the physical space of the venue based on the coordinatesof the lighting beam destination in the three-dimensional model space.The electronic processor controls actuation of a motor associated withthe lighting fixture to move the lighting fixture to a correspondinglighting fixture orientation in the physical space of the venue based onthe lighting fixture orientation value.

In some embodiments, the user device is associated with at least onecamera. The indicating, with the user device, the lighting beamdestination in the physical space of the venue using the position andthe orientation of the user device includes indicating the lighting beamdestination relative to a capture view of the at least one camera.

In some embodiments, the user device is associated with at least onecamera. The pairing of the user device in the venue with thethree-dimensional model space that represents the physical space of thevenue includes using a capture view of the at least one camera todetermine the position and the orientation of the user device.

In some embodiments, the pairing of the user device in the venue withthe three-dimensional model space that represents the physical space ofthe venue includes detecting a reference point in the venue with the atleast one camera associated with the user device.

In some embodiments, the user device displays an augmented reality imageof the venue based on the capture view of the at least one camera.

In some embodiments, the indicating of the lighting beam destination inthe physical space of the venue using the position and the orientationof the user device includes designating a location on a surface of thevenue where an area of the capture view of the camera meets the surfaceas the lighting beam destination.

In some embodiments, the controlling of actuation of the motorassociated with the lighting fixture to move the lighting fixture to thelighting fixture orientation in the physical space of the venue based onthe orientation values includes controlling, with the electronicprocessor, movement of the lighting fixture to follow the lighting beamdestination as the capture view of the camera changes.

In some embodiments, the indicating of the lighting beam destination inthe physical space of the venue using the position and the orientationof the user device includes detecting, by the electronic processor, apointing direction of an end of the user device in a lighting beamdestination direction in the physical space while the user device ispaired with the three-dimensional model space.

In some embodiments, at least one camera is associated with the userdevice such that a capture view of the camera faces a directionperpendicular to the pointing direction of the end of the user device.

In some embodiments, the controlling of the actuation of the motorassociated with the lighting fixture to move the lighting fixture to thecorresponding lighting fixture orientation includes controlling, withthe electronic processor, movement of the lighting fixture to follow thelighting beam destination as the capture view of the camera changes.

Systems are described herein for directing a lighting fixture in avenue. The system includes a lighting fixture, a motor associated withthe lighting fixture, and a user device. The user device includes acontroller that has an electronic processor and a memory. The memorystores instructions that when executed by the electronic processorconfigure the controller to pair the user device with athree-dimensional model space of a venue that represents a physicalspace of the venue. The controller indicates a lighting beam destinationin the physical space of the venue using a position and an orientationof the user device and converts the indicated lighting beam destinationinto coordinates in the three-dimensional model space based on theposition and orientation of the user device in the physical space of thevenue. The controller determines a lighting fixture orientation value todirect a lighting beam of the lighting fixture to the lighting beamdestination in the physical space of the venue based on the coordinatesof the lighting beam destination in the three-dimensional model space.The controller controls actuation of the motor associated with thelighting fixture to move the lighting fixture to a correspondinglighting fixture orientation in the physical space of the venue based onthe lighting fixture orientation value.

In some embodiments, the systems include at least one camera associatedwith the electronic processor. The controller is further configured toindicate the lighting beam destination relative to a capture view of theat least one camera.

In some embodiments, the systems include at least one camera associatedwith the electronic processor. The controller is further configured touse a capture view of the at least one camera to determine orientationand position of the user device.

In some embodiments, the controller is further configured to detect areference point in the venue with the at least one camera associatedwith the electronic processor.

In some embodiments, the user device displays an augmented reality imageof the venue based on the capture view of the at least one camera.

In some embodiments, the controller is further configured to designate alocation on a surface of the venue where a point or area of the captureview of the camera meets the surface as the lighting beam destination.

In some embodiments, the controller is further configured to controlmovement of the lighting fixture to follow the lighting beam destinationas the capture view of the camera changes.

In some embodiments, the controller is further configured to detect apointing direction of an end of the user device in a lighting beamdestination direction in the physical space while the user device ispaired with the three-dimensional model space.

In some embodiments, at least one camera is associated with the userdevice such that a capture view of the camera faces a directionperpendicular to the pointing direction of the end of the user device.

In some embodiments, the controller is further configured to controlmovement of the lighting fixture to follow the lighting beam destinationas the capture view of the camera changes.

In some embodiments, the lighting beam destination in the physical spaceof the venue is designated by receiving a distance and the orientationfrom the user device.

Before any embodiments are explained in detail, it is to be understoodthat the embodiments are not limited in its application to the detailsof the configuration and arrangement of components set forth in thefollowing description or illustrated in the accompanying drawings. Theembodiments are capable of being practiced or of being carried out invarious ways. Also, it is to be understood that the phraseology andterminology used herein are for the purpose of description and shouldnot be regarded as limiting. The use of “including,” “comprising,” or“having” and variations thereof are meant to encompass the items listedthereafter and equivalents thereof as well as additional items. Unlessspecified or limited otherwise, the terms “mounted,” “connected,”“supported,” and “coupled” and variations thereof are used broadly andencompass both direct and indirect mountings, connections, supports, andcouplings.

In addition, it should be understood that embodiments may includehardware, software, and electronic components or modules that, forpurposes of discussion, may be illustrated and described as if themajority of the components were implemented solely in hardware. However,one of ordinary skill in the art, and based on a reading of thisdetailed description, would recognize that, in at least one embodiment,the electronic-based aspects may be implemented in software (e.g.,stored on non-transitory computer-readable medium) executable by one ormore processing units, such as a microprocessor and/or applicationspecific integrated circuits (“ASICs”). As such, it should be noted thata plurality of hardware and software based devices, as well as aplurality of different structural components, may be utilized toimplement the embodiments. For example, “servers” and “computingdevices” described in the specification can include one or moreprocessing units, one or more computer-readable medium modules, one ormore input/output interfaces, and various connections (e.g., a systembus) connecting the components.

Other aspects of the embodiments will become apparent by considerationof the detailed description and accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a system for determining arrangement information of alighting fixture.

FIG. 1A illustrates another system for determining arrangementinformation of a lighting fixture.

FIG. 2 illustrates a controller for the system of FIG. 1.

FIG. 2A illustrates a controller for the system of FIG. 1A.

FIG. 3 illustrates cameras and lighting fixtures in a venue for thesystem of FIG. 1.

FIG. 3A illustrates cameras and lighting fixtures in a venue for thesystem of FIG. 1A.

FIG. 4 illustrates an application interface screen for use with thesystem of FIG. 1 and/or FIG. 1A for controlling the movement of alighting fixture according to user input.

FIG. 5 illustrates a scan of a surface a camera may detect to determinea centroid of a lighting beam.

FIG. 6 illustrates an application interface screen for use with thesystem of FIG. 1 and/or FIG. 1A for controlling the movement of alighting fixture according to a user input designating a lighting beamdestination.

FIG. 7 illustrates a process for determining a lighting fixturearrangement.

FIG. 8 illustrates a process for determining a lighting fixturearrangement.

FIG. 9 illustrates a process for directing a lighting fixture in avenue.

DETAILED DESCRIPTION

Embodiments described herein relate to accurately determiningarrangement information of one or more lighting fixtures and accuratelyfocusing one or more lighting fixtures on a lighting beam destination.Both of these tasks conventionally require skilled technicians, preciseand expensive measuring tools, and significant time. These tasks areachieved by acquiring arrangement information and subsequentlycontrolling lighting fixtures based on the arrangement information.

For example, FIG. 1 illustrates a system 100 for determining arrangementinformation of one or more lighting fixtures 102 and subsequentlydirecting the one or more lighting fixtures 102 in a venue 104 (shown inFIG. 3). The system 100 includes a user input device 106A-106D, acontrol board or control panel 108, lighting fixtures 102, cameras 110,a network 112, and a server-side computer or server 114. The user inputdevice 106A-106D includes, for example, a personal or desktop computer106A, a laptop computer 106B, a tablet computer 106C, or a mobile phone(e.g., a smart phone) 106D. Other user input devices include, forexample, an augmented reality headset or glasses. In some embodiments,the cameras 110 are integrated with the user input device 106A-106D,such as the camera of the mobile phone 106D. In other embodiments, thecameras 110 are separate from the user input device 106A-106D.

The user input device 106A-106D is configured to communicatively connectto the server 114 through the network 112 and provide information to, orreceive information from, the server 114 related to the control oroperation of the system 100. The user input device 106A-106D is alsoconfigured to communicatively connect to the control board 108 toprovide information to, or receive information from, the control board108. The connections between the user input device 106A-106D and thecontrol board 108 or network 112 are, for example, wired connections,wireless connections, or a combination of wireless and wiredconnections. Similarly, the connections between the server 114 and thenetwork 112, the control board 108 and the lighting fixtures 102, or thecontrol board 108 and the cameras 110 are wired connections, wirelessconnections, or a combination of wireless and wired connections.

The network 112 is, for example, a wide area network (“WAN”) (e.g., aTCP/IP based network), a local area network (“LAN”), a neighborhood areanetwork (“NAN”), a home area network (“HAN”), or personal area network(“PAN”) employing any of a variety of communications protocols, such asWi-Fi, Bluetooth, ZigBee, etc. In some implementations, the network 112is a cellular network, such as, for example, a Global System for MobileCommunications (“GSM”) network, a General Packet Radio Service (“GPRS”)network, a Code Division Multiple Access (“CDMA”) network, anEvolution-Data Optimized (“EV-DO”) network, an Enhanced Data Rates forGSM Evolution (“EDGE”) network, a 3GSM network, a 4GSM network, a 4G LTEnetwork, a 5G New Radio, a Digital Enhanced Cordless Telecommunications(“DECT”) network, a Digital AMPS (“IS-136/TDMA”) network, or anIntegrated Digital Enhanced Network (“iDEN”) network, etc.

FIG. 1A illustrates an alternative system 100A for determiningarrangement information of one or more lighting fixtures 102 andsubsequently controlling the lighting fixtures 102. The hardware of thealternative system 100A is identical to the above system 100, except thecontrol board or control panel 108 is removed. As such, the user inputdevice 106A-106D is configured to communicatively connect to thelighting fixtures 102 and to the cameras 110. The connections betweenthe user input device 106A-106D and the lighting fixtures 102 and theconnections between the user input device 106A-106D and the camera 110are wired connections, wireless connections, or a combination ofwireless and wired connections.

FIG. 2 illustrates a controller 200 for the system 100. The controller200 is electrically and/or communicatively connected to a variety ofmodules or components of the system 100. For example, the illustratedcontroller 200 is connected to one or more indicators 202 (e.g., LEDs, aliquid crystal display [“LCD”], etc.), a user input or user interface204 (e.g., a user interface of the user input device 106A-106D in FIG.1), and a communications interface 206. The controller 200 is alsoconnected to the control board 108. The communications interface 206 isconnected to the network 112 to enable the controller 200 to communicatewith the server 114. The controller 200 includes combinations ofhardware and software that are operable to, among other things, controlthe operation of the system 100, control the operation of the lightingfixture 102, control the operation of the camera 110, receive one ormore signals from the camera 110, communicate over the network 112,communicate with the control board 108, receive input from a user viathe user interface 204, provide information to a user via the indicators202, etc. In some embodiments, the indicators 202 and the user interface204 are integrated together in the form of, for instance, atouch-screen.

In the embodiment illustrated in FIG. 2, the controller 200 isassociated with the user input device 106A-106D. As a result, thecontroller 200 is illustrated in FIG. 2 as being connected to thecontrol board 108 which is, in turn, connected to the lighting fixtures102 and the cameras 110. In other embodiments, the controller 200 isincluded within the control board 108, and, for example, the controller200 can provide control signals directly to the lighting fixtures 102and the cameras 110. In other embodiments, the controller 200 isassociated with the server 114 and communicates through the network 112to provide control signals to the control board 108, the lightingfixtures 102, and/or the cameras 110.

The controller 200 includes a plurality of electrical and electroniccomponents that provide power, operational control, and protection tothe components and modules within the controller 200 and/or the system100. For example, the controller 200 includes, among other things, aprocessing unit 208 (e.g., a microprocessor, a microcontroller, oranother suitable programmable device), a memory 210, input units 212,and output units 214. The processing unit 208 includes, among otherthings, a control unit 216, an arithmetic logic unit (“ALU”) 218, and aplurality of registers 220 (shown as a group of registers in FIG. 2),and is implemented using a known computer architecture (e.g., a modifiedHarvard architecture, a von Neumann architecture, etc.). The processingunit 208, the memory 210, the input units 212, and the output units 214,as well as the various modules or circuits connected to the controller200 are connected by one or more control and/or data buses (e.g., commonbus 222). The control and/or data buses are shown generally in FIG. 2for illustrative purposes. The use of one or more control and/or databuses for the interconnection between and communication among thevarious modules, circuits, and components would be known to a personskilled in the art in view of the embodiments described herein.

The memory 210 is a non-transitory computer readable medium andincludes, for example, a program storage area and a data storage area.The program storage area and the data storage area can includecombinations of different types of memory, such as a ROM, a RAM (e.g.,DRAM, SDRAM, etc.), EEPROM, flash memory, a hard disk, an SD card, orother suitable magnetic, optical, physical, or electronic memorydevices. The processing unit 208 is connected to the memory 210 andexecutes software instructions that are capable of being stored in a RAMof the memory 210 (e.g., during execution), a ROM of the memory 210(e.g., on a generally permanent basis), or another non-transitorycomputer readable medium such as another memory or a disc. Softwareincluded in the implementation of the system 100 and controller 200 canbe stored in the memory 210 of the controller 200. The softwareincludes, for example, firmware, one or more applications, program data,filters, rules, one or more program modules, and other executableinstructions. The controller 200 is configured to retrieve from thememory 210 and execute, among other things, instructions related to thecontrol processes and methods described herein. In other embodiments,the controller 200 includes additional, fewer, or different components.

The user interface 204 is included to provide user control of the system100, the lighting fixtures 102, and/or the cameras 110. The userinterface 204 is operably coupled to the controller 200 to control, forexample, control or drive signals provided to the lighting fixtures 102and/or control or drive signals provided to the cameras 110. The userinterface 204 can include any combination of digital and analog inputdevices required to achieve a desired level of control for the system100. For example, the user interface 204 can include a computer having adisplay and input devices, a touch-screen display, a plurality of knobs,dials, switches, buttons, faders, or the like. In the embodimentillustrated in FIG. 2, the user interface 204 is separate from thecontrol board 108. In other embodiments, the user interface 204 isincluded in the control board 108.

The controller 200 is configured to work in combination with the controlboard 108 to provide direct control or drive signals to the lightingfixtures 102 and/or the cameras 110. As described above, in someembodiments, the controller 200 is configured to provide direct controlor drive signals to the lighting fixtures 102 and/or the cameras 110without separately interacting with the control board 108 (e.g., thecontrol board 108 includes the controller 200). The direct drive signalsthat are provided to the lighting fixtures 102 and/or the cameras 110are provided, for example, based on a user input received by thecontroller 200 from the user interface 204. The controller 200 is alsoconfigured to receive one or more signals from the cameras 110 relatedto image or scan data.

As shown in FIG. 2A and described above, the system 100A includes thecontroller 200 configured to work without the control board 108, suchthat the controller 200 is configured to provide signals to the lightingfixtures 102 and/or the cameras 110 and to receive one or more signalsfrom the cameras 110 related to image or scan data.

FIG. 3 illustrates the control board 108, the lighting fixture 102, thecamera 110, and the user input device 106A-106D of the system 100 in thevenue 104. The user input device 106A-106D directs the lighting fixture102 such that a lighting beam 300 projecting from the lighting fixture102 strikes at discrete locations 302A, 302B, 302C, 302D on a stagesurface 304 at the venue 104. A user may directly control the movementof the lighting fixture 102, or the lighting fixture 102 may moveaccording to a preprogrammed pattern.

FIG. 3A illustrates the system 100A in the venue 104. As describedabove, the system 100A removes the control board 108, and the user inputdevice 106A-106D is configured to directly communicate with the lightingfixture 102 and the camera 110.

With reference to the system 100 and/or the system 100A, FIG. 4illustrates an example of an application interface screen 400 for usewith the user device 106A-106D that receives user input to control themovement of the lighting fixture 102 for synchronizing the position ofthe lighting beam 300 with the discrete locations 302 on the ground inthe venue 104. In some embodiments, the lighting beam 300 moves to atleast three locations (302A, 302B, 302C). Other embodiments include thelighting beam 300 moving to a fourth location 302D. Other embodimentsinclude the lighting beam 300 moving to more than four locations 302.The movement of the lighting fixture 102 is accomplished by changing theangle of the lighting fixture 102 by either panning or tilting thelighting fixture 102. The controller 200 is configured to store theangular change data corresponding to the lighting fixture 102 movementto move the lighting beam 300 from the first location 302A to the secondlocation 302B, from the second location 302B to the third location 302C,and so on.

With reference to FIGS. 3 and 3A, the controller 200 is furtherconfigured to store the coordinate data of each of the at least threelocations 302 on the surface 304. In some embodiments, the coordinatedata is input by a user, such as when the user directly controls themovement of the lighting fixture 102. In some embodiments, multiplelocations 302 are plotted in a model, and the user indicates a desiredlighting beam destination 308 by, for example, standing at one of thelocations 302 while holding the user device 106A-106D. Once the lightingbeam 300 is projecting onto the desired lighting beam destination 308that is also one of the locations 302, the user can designate thepresent lighting beam destination 308 as corresponding to the particularlocation 302 by, for instance, selecting the appropriate one of thelocations 302 from a list.

In some embodiments, the coordinate data is determined by the controller200 by calculating a position of the user device 106A-106D relative toone or more reference points 306 with scan data from one or more cameras110. The cameras 110 may be integrated into the user device 106A-106D,wirelessly connected to the user device 106A-106D, connected by wire tothe user device 106A-106D, or otherwise associated. The reference points306 provide orientation and distance information for the user device106A-106D. In some embodiments, the reference points 306 are visiblemarks on the surface 304. Other embodiments include at least onereference point 306 in the form of a sensor readable marker that is notvisible to the human eye (e.g., an infrared marker). Using knowncomputer vision, image recognition, and scanning applications (e.g., asimultaneous localization and mapping [“SLAM”] program), the controller200 can calculate distances between designated points on the surface 304after the user device 106A-106D has been properly calibrated with thereference points 306.

To determine the discrete locations 302 where the lighting beam 300contacts the surface 304 without user input information regarding thelocations, the controller 200 is configured to determine a centroid ofthe lighting beam through scan data provided by the camera 110. Anexample of the scan of the surface 304 that the camera 110 may performis shown in FIG. 5. The centroid can be found regardless of angle ofattack of the lighting beam 300 through any appropriate methodincluding, for example, light intensity analysis of the surface 304. Assuch, at each of the discrete locations 302, the image data of thelighting beam 300 is captured by the camera 110 and analyzed by thecontroller 200. Once the analysis is complete, the controller 200 isconfigured to return values for the coordinate data of each of thediscrete locations 302 relative to the one or more reference points 306.

Because the lighting fixture 102 control is paired with the controller200, the controller 200 is able to quantify the change in angle eachtime the lighting fixture 102 moves. Although this change in angle isknown to the controller 200 as a relative angle of the lighting fixture102 from one position to another and not an absolute angle relative tothe surface 304, the absolute angles can be found through mathematicalcalculations using a perspective inversion solution described generallybelow.

To calculate the position of the lighting fixture 102 relative to thestage surface 304, the perspective inversion solution uses the length ofeach side of a triangle that is traced by the lighting beam 300 on thestage surface 304 and the changes in angle of the lighting fixture 102that created that triangle. The length of the sides of the triangle canbe found with the at least three locations 302 coordinate data inputand/or calculation as described above. The angles are known by virtue ofthe controller 200 controlling the lighting fixture 102, as describedabove.

Because there can be a degree of uncertainty present when calculatingthe position of the lighting fixture 102 based on only three discretelocations 302A, 302B, and 302C, some embodiments include a fourthdiscrete location 302D. With four discrete locations 302A, 302B, 302C,302D, the controller 200 is configured to sequentially determine sets ofthree discrete locations (e.g., 302A, 302B, and 302C first, 302B, 302C,and 302D second, 302A, 302C, and 302D third, etc.) and is configured toreturn a value for the lengths of the lighting beam 300 as it existedwhen it was directed to each of the discrete locations 302A, 302B, 302C,302D. The controller 200 is then configured to compare these results asthey overlap in order to calculate the values with greater certainty.Other embodiments include more than the four discrete locations 302.Such embodiments add even further accuracy to the calculation. Once thelength of the lighting beam 300 from the lighting fixture 102 to eachindividual discrete location 302A, 302B, 302C, 302D is found, thecontroller 200 is configured to, for example, trilaterate orquadrilaterate the location of the lighting fixture 102. The point atwhich the spheres of possible solutions for the discrete locations 302A,302B, 302C, 302D cross is designated as the location of the lightingfixture 102. This calculation actually returns two results-one above thestage surface 304 and one below the stage surface 304. The controller200 is configured to discard the result below the stage surface 304.

In some embodiments of the system 100 and/or the system 100A, thecontroller 200 is further configured to run an optimizer operation withthe possible positions of the lighting fixture 102. Because themeasurements could be off slightly or the control feedback may havenoise in the signal, an optimizer operation can more accuratelydetermine the position of the lighting fixture 102 (e.g., improveaccuracy of the position of the lighting fixture). The optimizer runscalculations using the law of cosines with the values it has frompreviously running the perspective inversion solution. The optimizertakes the length of the lighting beam 300 from the lighting fixture 102to each individual discrete location 302A, 302B, 302C, 302D, combinesthat data with the known changes in angle of the lighting fixture 102,and determines possible values for the distances on the stage surface304 between the discrete locations 302A, 302B, 302C, 302D. Because thesedistances are known through measurement or other methods describedabove, the optimizer compares these known distances with the determineddistances to gauge the accuracy of the results from the perspectiveinversion solution.

An example of an appropriate optimizer operation is a limited memoryBroyden-Fletcher-Goldfarb-Shanno (“LBFGS”) optimizer, although otheroptimizer operations may be used. If the optimizer operation returnsresults that converge to a value, that particular value is determined tobe more accurate than the initial value. If the results do not convergeto a value and instead scatter, the initial value is returned asaccurate enough to continue without further attempting the optimizeroperation. After these steps, the location of the lighting fixture 102is again trilaterated (or quadrilaterated). This location is then outputas the most accurate estimation of the position of the lighting fixture102 relative to the stage surface 304 (or the reference points 306).

After the controller 200 has determined the position of the lightingfixture 102, the controller 200 is configured to determine theorientation of the lighting fixture 102 relative to the stage surface304. In some embodiments, however, the position calculation for thelighting fixture 102 and the orientation calculation for the lightingfixture 102 are both accomplished with the optimizer operation.

The controller 200 uses any three of the discrete locations 302 on thestage surface 304 and the corresponding relative angular changeinformation from the control of the lighting fixture 102. The relativeangular change information includes pan, tilt, or both pan and tilt. Thecontroller 200 determines spherical coordinates of the discretelocations 302 receiving the lighting beam 300 as the lighting fixture102 is oriented in each position. These spherical coordinates arerelative spherical coordinates, in that they include pan and tilt anglesof the lighting fixture 102 relative to the axis of the lighting beam300, and the origin is the position of the lighting fixture 102 (i.e.,the focal point of the lighting beam 300).

The controller 200 is configured to translate the known Cartesiancoordinates of the found position of the lighting fixture 102 and theknown discrete locations 302 relative to the reference points 306 intoreal-world spherical coordinates with the lighting fixture 102 as theorigin. Some embodiments include the reference points 306 being one ofthe known discrete locations 302 in this calculation.

The controller 200 is then configured to perform a matrix transformationutilizing both the relative spherical coordinates and the real-worldspherical coordinates to translate the relative spherical coordinates ofthe orientation of the lighting fixture 102 at each position intoreal-world spherical coordinates (e.g. relative to a reference plane,which may be referred to as absolute spherical coordinates). Once thisrelationship is determined, the yaw, pitch, and roll information of theorientation of the lighting fixture 102 relative to the stage surface304 is extracted. In some embodiments, the yaw, pitch, and roll may bereferred to as absolute angles of the lighting fixture 102 withreference to the surface 304, which includes a plane of the discretelocations 302A, 302B, 302C, and 302D. This information is the absoluteorientation of the lighting fixture 102 regardless of mounting methods.

After the above calculations have been completed, the controller 200 isconfigured to present the results as the indicated position andorientation of the lighting fixture 102 (e.g., the controller 200, or auser device 106A-106D is paired with the three-dimensional model spaceof the venue). With this information, the controller 200 can alter imagedata relating to the lighting fixture 102 and the lighting beam 300 inan interactive environment and control the lighting fixture 102. Oncethe lighting fixtures 102 in the venue 104 have been identified,classified, and located, the above calculated information can be used toimplement transitions of various styles.

It should be noted, however, that the three-dimensional model space ofthe venue may, in some embodiments, be rendered for observation by theuser, but, in other embodiments, may instead be used for internalcalculations by the system 100, 100A without producing a rendering ofthe model for user observation or interaction.

With continued reference to FIGS. 3 and 3A, the above calculatedinformation can also be used to alter command string data sent to thelighting fixture 102 in order to translate locations 308 designated onthe surface 304 into appropriate angular changes of the lighting fixture102 to cause the lighting beam 300 to be directed to the designatedlocations 308. Some embodiments of the system 100, 100A include thecontroller 200 configured to control the lighting fixture 102 accordingto the altered command string data.

In some embodiments, the indication of the locations 308 is made on atouchscreen of the user device 106A-106D utilizing an augmented realityinterface (through, for instance, an application interface screen 600 asshown in FIG. 6). In such an interface, the user sees the surface 304 onthe touchscreen and may point to a destination 308 on the surface 304 onthe touchscreen. The controller 200 is configured to then convert thisindicated portion of the screen into an equivalent position of thedestination 308 on the surface 304. The controller 200 is configured torelate the orientation of the capture view of the camera 110 with thesurface 304 based on a calibration with one or more reference points306.

Additionally or alternatively, the system 100, 100A uses one or moreinertial measurement units (“IMUs”) coupled with the user device106A-106D to determine the position and orientation data of the userdevice 106A-106D. Cameras 110 may not be necessary in this instance, butthe user device 106A-106D would be paired to the three-dimensional modelspace by positioning and orienting the device in a known homearrangement and recording the data from the IMUs at that homearrangement. In embodiments of the system 100, 100A using augmentedreality libraries (e.g., ARCore, ARKit, etc.), both IMUs and cameras 110can be utilized to improve accuracy of the data.

In some embodiments, the system 100, 100A uses one or more sensors (oneexample of which being the cameras 110) to determine the position of theuser device 106A-106D. As stated above, the one or more sensors caninclude cameras 110 and/or IMUs. Additionally or alternatively, the oneor more sensors can include a Wi-Fi, Bluetooth, ZigBee, or other systemhaving one or more proximity beacons. Other example sensors can includeinfrared cameras, thermal cameras, depth cameras, ultra-wideband (UWB)radar sensors, or the like to determine focus points of the lightingbeam 300 at each location 302.

Once the real-world position of the destination 308 on the surface 304is determined, the controller 200 is configured to send a control signalto one or more motors to actuate movement of the lighting fixture 102.The lighting fixture 102 moves to the appropriate orientation to projectthe lighting beam 300 at the destination 308. For example, thecontroller 200 is configured to translate the real-world Cartesiancoordinates of the destination 308 into the altered control stringdescribed above to operate the lighting fixture 102 such that thelighting beam 300 moves appropriately in the three-dimensional modelspace.

In some embodiments of the system 100, 100A, the indication of thedesired destination 308 for the lighting beam 300 on the surface 304 atthe venue 104 can be made by aiming the center of the capture view ofthe camera 110 at the destination 308. As described above, thecontroller 200 is configured to convert this center of the capture viewinto an equivalent position of the destination 308 on the actual surface304. In this configuration, the indication of the desired destination308 may be actuated by a distinct command, such as a voice command, thepress of a button, or the like. Additionally or alternatively, theindication of the desired destination 308 is switched to a continual orcontinuous mode, such that the desired destination 308 movessimultaneously or with some delay relative to the changing capture viewof the camera 110 as the camera 110 is moved throughout the venue 104.In some embodiments, this mode can be used as a follow spot control.

In some embodiments of the system 100, 100A, the indication of thedesired destination 308 of the lighting beam 300 on the surface 304 atthe venue 104 is made by pointing an end of the user device 106A-106D ina direction with the camera view of the camera 110 pointing in anorthogonal direction. With a smartphone 106D, for instance, a user couldpoint the top end of the smartphone 106D at the desired location 308while the camera 110 is directed toward the surface 304. In thisconfiguration, the lighting beam destination 308 may be set at aconstant distance, potentially designated by the user, from the end ofthe smartphone 106D or from the center of the capture view of the camera110 in an orthogonal direction from the direction of the capture view.In some embodiments, the user device 106A-106D determines the locationof the desired destination 308 by pointing the end of the user device106A-106D to the desired destination 308, and using the known location(coordinates) of the user device 106A-106D in the venue along with atilting angle of the device 106A-106D relative to the surface 304 (e.g.,determined using internal IMUS of the device 106A-106D) to determine thelocation of the of the desired destination 308 in the venue 104.

In some embodiments of the system 100, 100A, the indication of thedesired destination 308 of the lighting beam 300 is set as the locationof the user device 106A-106D itself. The controller 200 determines thelocation of the user device 106A-106D based on the capture data from thecamera 110. This data is processed to calculate the location relative toone or more reference points 306. The controller 200 is configured todesignate the current location of the user device 106A-106D relative tothe reference points 306 as the destination 308. As described above, theindication of the desired destination 308 as the location of the userdevice 106A-106D can be actuated by a distinct command. Additionally oralternatively, the indication of the user device 106A-106D as thedestination 308 may be switched to a continuous or continual mode.

As shown in FIG. 7, the system 100, 100A may operate according to amethod 700 to calculate the arrangement information of the lightingfixture 102. First, the user chooses and measures four discrete physicallocations 302A, 302B, 302C, 302D on the surface 304 (STEP 701).

The user then focuses the lighting fixture 102 at each of the fourdiscrete locations 302A, 302B, 302C, 302D and saves the resultingangular change values for the pan and tilt of the lighting fixture (STEP702). Next, either the controller 200 or the user selects any three ofthe four discrete locations 302A, 302B, 302C, 302D and the correspondingangular changes the lighting fixture 102 made to direct the lightingbeam 300 to each of the respective selected discrete locations 302A,302B, 302C, 302D (STEP 703).

A perspective inversion solution is used to solve for the distances fromthe discrete locations 302A, 302B, 302C, 302D on the surface 304 to thelighting fixture 102 (STEP 704). Once all the values for the distanceshave been determined, the position of the lighting fixture 102 istrilaterated (STEP 705).

The controller 200 then determines whether all of the possiblecombinations of three of the discrete locations 302A, 302B, 302C, 302Dand corresponding angular changes have been calculated with theperspective inversion solution (STEP 706). If not all possiblecombinations have been calculated, the method 700 returns to STEP 703 tocomplete the other possible combinations.

If, at STEP 706, all possible combinations have been calculated, theprocess 700 proceeds to compute an error of each possible solution found(STEP 707). Next, the controller 200 saves the solution with the fewesterrors as the best initial solution for the position of the lightingfixture 102 (STEP 708). The best initial solution is then used as aninput to attempt to optimize (e.g., improve accuracy of) the result byrunning calculations using the law of cosines (STEP 709). The controller200 then determines whether the optimization operation converged on asolution (STEP 710).

If the optimization operation converged on a solution, the optimalsolution is returned as the solution for the length of the light beam300 from each of the discrete locations 302A, 302B, 302C, 302D to thelighting fixture 102 (STEP 711A) instead of the previous best initialsolution from STEP 708. If the optimization operation did not convergeon a solution, the controller 200 ignores the optimization operation andreturns the best initial solution from STEP 708 (STEP 711B). Thecontroller 200 then determines the position of the lighting fixture 102through trilateration with the best available lengths (STEP 712).

Now that the position of the lighting fixture 102 has been determined,the controller 200 selects one set of three of the discrete locations302 and the corresponding changes in angle of the lighting fixture 102(STEP 713). The spherical coordinates of the discrete locations 302 arefound with the lighting fixture 102 serving as the point of origin (STEP714). Then, the known Cartesian coordinates of the discrete locations302 and the lighting fixture 102 are converted to real-world sphericalcoordinates (STEP 715) with the lighting fixture 102 as the origin. Amatrix transformation is performed to translate the relative sphericalcoordinates of the lighting fixture 102 into absolute sphericalcoordinates (STEP 716). The yaw, pitch, and roll information of thelighting fixture 102 is then determined and extracted (STEP 717). Thecontroller 200 then returns the position and orientation of the lightingfixture 102 relative to the surface 304 and the reference point 306(STEP 718).

Although STEPS 713-717 were described above, some embodiments of themethod 700 includes the position calculation for the lighting fixture102 and the orientation calculation for the lighting fixture 102 bothbeing accomplished during the optimization step (STEP 709) andproceeding from STEP 712 directly to STEP 718.

With reference to FIG. 8, the system 100, 100A may additionally oralternatively operate according to a method 800 to calculate thearrangement information of the lighting fixture 102. First, the lightingfixture 102 is turned on (STEP 801). A control routine is operated, andthe controller 200 records the set angle of the lighting fixture 102while the camera 110 captures the discrete location 302 of the lightingbeam 300 on the surface 304 at three arbitrary points (STEP 802). Thecontroller 200 then calculates the distances from the discrete locations302 to the lighting fixture 102 (STEP 803). These distances are used totrilaterate the position of the lighting fixture 102 (STEP 804).

The method 800 then moves to STEP 805, where the error of each possiblesolution is calculated. The controller 200 saves the solution with theleast errors as the best initial solution for the position of thelighting fixture 102 (STEP 806). The best initial solution is used as aninput to attempt to optimize the result by running calculations usingthe law of cosines (STEP 807). The controller 200 then determineswhether the initial solution (after optimization) for the position ofthe lighting fixture 102 is known with enough accuracy to be below anerror threshold (STEP 808).

If the position error is not less than the error threshold at STEP 808,the controller 200 determines whether the number of discrete locations302 recorded by a positions counter is above a threshold value (STEP809). The threshold positions value may be any appropriate numberincluding, for instance, ten discrete locations 302. If, at STEP 809,the positions counter is less than the threshold value, the controller200 moves the lighting fixture 102 to a new angular position (STEP 810)and increases the value stored in the positions counter by one. Next,the controller 200 captures data corresponding to another discretelocation 302 (STEP 811). After capturing the data corresponding toanother discrete location 302 (STEP 811), the method 800 returns to STEP803 to recalculate the distances from the discrete locations 302 to thelighting fixture 102. The method 800 continues through STEPS 804-807.

This portion of the method 800 loops until either the initial solution(after optimization) is found within the error threshold or the numberstored in the positions counter is above the threshold value. In someembodiments, the addition of the fourth discrete location 302D makes theinitial solution fall within the error threshold. In other embodiments,five or more discrete locations 302 are used. In other embodiments, onlythe initial three discrete locations 302A, 302B, and 302C are used toget an initial solution that is within the error threshold. If, at STEP808, position error is less than or equal to the error threshold, themethod 800 continues to STEP 812. Similarly, if the new initial solutionfound at STEP 806 is sufficiently accurate after optimization and afterthe method 800 has continued through the loop of STEPS 807-811 and803-808, the method 800 continues to STEP 812. Further, if the initialsolution found at STEP 806 and optimized at STEP 807 is not within theerror threshold but the positions counter has a value that is above thepositions threshold, the method 800 continues to STEP 812 without tryingfurther discrete locations 302.

The controller 200 then determines whether the optimization operationconverged on a solution (STEP 812). If the optimization operationconverged on a solution, the optimal solution is returned as thesolution for the lengths of the light beam 300 from each of the discretelocations 302 to the lighting fixture 102 (STEP 813A) instead of theprevious best initial solution from STEP 806. If the optimizationoperation did not converge on a solution, the controller 200 ignores theoptimization operation and returns the best initial solution from STEP806 (STEP 813B). The controller 200 then calculates the position of thelighting fixture 102 for a final time through trilateration with thebest available values for the lengths from the discrete locations 302 tothe lighting fixture 102 (STEP 814).

With the position of the lighting fixture 102 determined, the controller200 selects one set of three of the discrete locations 302 and thecorresponding changes in angle of the lighting fixture 102 (STEP 815).The spherical coordinates of the discrete locations 302 are found withthe lighting fixture 102 serving as the point of origin (STEP 816).Then, the known Cartesian coordinates of the discrete locations 302 andthe lighting fixture 102 are converted to real-world sphericalcoordinates (STEP 817) with the lighting fixture 102 as the origin. Amatrix transformation is performed to translate the relative sphericalcoordinates of the lighting fixture 102 into absolute sphericalcoordinates (STEP 818). The yaw, pitch, and roll information of thelighting fixture 102 is then found and extracted (STEP 819). Thecontroller 200 then determines the position and orientation of thelighting fixture 102 relative to the surface 304 and the reference point306 (STEP 820).

Although STEPS 815-819 were described above, some embodiments of themethod 800 include the position calculation for the lighting fixture 102and the orientation calculation for the lighting fixture 102 both beingaccomplished during the optimization step (STEP 807) and proceeding fromSTEP 814 directly to STEP 820.

With reference to FIG. 9, a method 900 of directing a lighting fixture102 in the venue 104 is shown. The system 100, 100A may additionally oralternatively operate according to the method 900. The method 900 beginswith pairing the user device 106A-106D in the venue 104 with athree-dimensional model space of the lighting beam 300 and lightingfixture 102 (STEP 901). This step is accomplished, for instance, bydirecting the camera 110 such that the capture view of the camera scansat least one of the reference points 306. Once the reference points 306have been scanned, the controller 200 can determine where the userdevice 106A-106D is in the venue 104 and what orientation it has in thevenue 104 (e.g., as described above with respect to FIGS. 3 and 3A).

The method 900 also includes the controller 200 indicating a lightingbeam destination 308 (STEP 902). The lighting beam destination 308 maybe designated in, for instance, one of the ways described above. Thelighting beam destination 308 is located relative to the capture view ofthe camera 110. Once the lighting beam destination 308 has beenindicated, the method 900 includes the controller 200 converting thedestination indicated by the user device 106 into coordinates at thevenue 104 in the three-dimensional model space (STEP 903). Thisconversion is made based on the earlier gathered data about theorientation and position of the user device 106A-106D.

After this conversion has been made, the method 900 includes thecontroller 200 interpreting the coordinates at the venue 104 for thelighting beam destination 308 relative to lighting fixture arrangement(e.g., positions and orientations), and determining a correspondinglighting fixture 102 arrangement (e.g., using process 700 or process800) that directs the lighting beam 300 appropriately to the lightingbeam destination 308 (STEP 904). The method 900 then includes thecontroller 200 controlling actuation of at least one motor coupled to orassociated with the lighting fixture 102 to move the lighting fixture102 according to the determined lighting fixture 102 orientation suchthat the lighting beam 300 is directed to the lighting beam destination308 (STEP 905).

Thus, embodiments described herein provide methods and systems fordirecting a lighting fixture in a venue. Various features and advantagesof some embodiments are set forth in the following claims.

What is claimed is:
 1. A method of directing a lighting fixture in a venue, the method comprising: pairing a user device with a three-dimensional model space that represents a physical space of the venue; indicating, with the user device, a lighting beam destination in the physical space of the venue using a position and an orientation of the user device in the physical space of the venue; converting, with an electronic processor, the indicated lighting beam destination into coordinates in the three-dimensional model space; determining, with the electronic processor, a lighting fixture orientation value to direct a lighting beam of the lighting fixture to the lighting beam destination in the physical space of the venue based on the coordinates of the lighting beam destination in the three-dimensional model space; and controlling, with the electronic processor, actuation of a motor associated with the lighting fixture to move the lighting fixture to a lighting fixture orientation in the physical space of the venue based on the lighting fixture orientation value.
 2. The method of claim 1, wherein the user device is associated with at least one camera, and wherein indicating, with the user device, the lighting beam destination in the physical space of the venue using the position and orientation of the user device includes: indicating the lighting beam destination relative to a capture view of the at least one camera.
 3. The method of claim 1, wherein the user device is associated with at least one camera, and wherein the pairing of the user device in the venue with the three-dimensional model space that represents the physical space of the venue includes: using a capture view of the at least one camera to determine the position and the orientation of the user device.
 4. The method of claim 2, wherein the pairing of the user device in the venue with the three-dimensional model space that represents the physical space of the venue includes: detecting a reference point in the venue with the at least one camera associated with the user device.
 5. The method of claim 2, wherein the user device displays an augmented reality image of the venue based on the capture view of the at least one camera.
 6. The method of claim 5, wherein the indicating, with the user device, the lighting beam destination in the physical space of the venue using the position and the orientation of the user device includes: designating a location on a surface of the venue where an area of the capture view of the camera meets the surface as the lighting beam destination.
 7. The method of claim 6, wherein the controlling, with the electronic processor, actuation of the motor associated with the lighting fixture to move the lighting fixture to the lighting fixture orientation in the physical space of the venue based on the orientation values includes: controlling, with the electronic processor, movement of the lighting fixture to follow the lighting beam destination as the capture view of the camera changes.
 8. The method of claim 1, wherein the indicating, with the user device, the lighting beam destination in the physical space of the venue using the position and the orientation of the user device includes: detecting, by the electronic processor, a pointing direction of an end of the user device in a lighting beam destination direction in the physical space while the user device is paired with the three-dimensional model space.
 9. The method of claim 8, wherein at least one camera is associated with the user device such that a capture view of the camera faces a direction perpendicular to the pointing direction of the end of the user device.
 10. The method of claim 9, wherein the controlling, with the electronic processor, actuation of the motor associated with the lighting fixture to move the lighting fixture to the corresponding lighting fixture orientation includes: controlling, with the electronic processor, movement of the lighting fixture to follow the lighting beam destination as the capture view of the camera changes.
 11. The method of claim 1, wherein the lighting beam destination in the physical space of the venue is designated by receiving, by the electronic processor, a distance and the orientation from the user device.
 12. A system comprising: a lighting fixture; a motor associated with the lighting fixture; a user device including a controller, the controller including an electronic processor and a memory, the memory storing instructions that when executed by the electronic processor configure the controller to: pair the user device with a three-dimensional model space of a venue that represents a physical space of the venue, indicate a lighting beam destination in the physical space of the venue using a position and an orientation of the user device in the physical space of the venue, convert the indicated lighting beam destination into coordinates in the three-dimensional model space based on the position and orientation of the user device in the physical space of the venue, determine a lighting fixture orientation value to direct a lighting beam of the lighting fixture to the lighting beam destination in the physical space of the venue based on the coordinates of the lighting beam destination in the three-dimensional model space; and control actuation of the motor associated with the lighting fixture to move the lighting fixture to a lighting fixture orientation in the physical space of the venue based on the lighting fixture orientation value.
 13. The system of claim 12, further comprising at least one camera associated with the electronic processor, wherein the controller is further configured to use a capture view of the at least one camera to determine the position and the orientation of the user device.
 14. The system of claim 12, further comprising at least one camera associated with the electronic processor, wherein the controller is further configured to indicate the lighting beam destination relative to a capture view of the at least one camera.
 15. The system of claim 14, wherein the controller is further configured to detect a reference point in the venue with the at least one camera associated with the electronic processor.
 16. The system of claim 14, wherein the user device displays an augmented reality image of the venue based on the capture view of the at least one camera.
 17. The system of claim 16, wherein the controller is further configured to designate a location on a surface of the venue where a point or area of the capture view of the camera meets the surface as the lighting beam destination.
 18. The system of claim 12, wherein the controller is further configured to control movement of the lighting fixture to follow the lighting beam destination as the capture view of the camera changes.
 19. The system of claim 12, wherein the controller is further configured to detect a pointing direction of an end of the user device in a lighting beam destination direction in the physical space while the user device is paired with the three-dimensional model space.
 20. The system of claim 19, wherein at least one camera is associated with the user device such that a capture view of the camera faces a direction perpendicular to the pointing direction of the end of the user device.
 21. The system of claim 20, wherein the controller is further configured to control movement of the lighting fixture to follow the lighting beam destination as the capture view of the camera changes.
 22. The system of claim 12, wherein the lighting beam destination in the physical space of the venue is designated by receiving a distance and the orientation from the user device. 